Smart guide to capture digital images that align with a target image model

ABSTRACT

The present disclosure includes systems, methods, and non-transitory computer readable media that can guide a user to align a camera feed captured by a user client device with a target digital image. In particular, the systems described herein can analyze a camera feed to determine image attributes for the camera feed. The systems can compare the image attributes of the camera feed with corresponding target image attributes of a target digital image. Additionally, the systems can generate and provide instructions to guide a user to align the image attributes of the camera feed with the target image attributes of the target digital image.

BACKGROUND

With advances in camera technology and the rise of camera-enabled mobiledevices, users are capturing more digital images than ever. The vastmajority of users have little or no training in professionalphotography, and as a result, many digital images are poor in quality.For example, many digital images have bad lighting or improper cameraangles that result in images that are unclear or that are otherwiseaesthetically unappealing. Accordingly, many camera-enabled mobiledevices provide applications that analyze and modify digital images,post capture, to improve the visual aesthetics of the images. Forexample, conventional systems utilize image filters or other imagingtools to alter brightness, contrast, colors, or other image attributesin a post hoc manner. While these conventional systems can modifydigital images to improve the overall quality of the images, thesesystems nonetheless suffer from a number of disadvantages.

For example, while conventional processes and systems can modify digitalimages to an extent, these systems nonetheless produce digital imagesthat are lower in quality than professional photographs. Indeed, becausethese conventional systems can only improve digital images based on theunderlying originally-captured images, resulting modified images areoften still low-quality.

In addition, conventional systems typically require a relatively largeamount of processing power. In particular, conventional systemstypically require processing time after capturing a digital image toanalyze and modify a digital image. For example, conventional systemsmay require a relatively large amount of computational resources, andthus, not be available on various types of mobile devices withrelatively limited computational resources. Additionally, the storage oforiginal and modified images can use large of amounts of storage space,particularly when a user takes many photos in an effort to capture ahigher quality image.

Thus, there are disadvantages with regard to conventional digital imagemodification systems.

SUMMARY

This disclosure describes one or more embodiments of methods,non-transitory computer readable media, and systems that solve theforegoing problems in addition to providing other benefits. While thissummary refers to systems for simplicity, the summary also applies tocertain disclosed methods and non-transitory computer readable media. Tosolve the foregoing and other problems, the disclosed systems analyze acamera feed associated with a camera of a user client device (e.g., asmartphone). The systems further determine image attributes associatedwith the camera feed and compare those image attributes withcorresponding image attributes of a target or template image. Based onthe comparison, the systems provide a guide to instruct the user of theuser client device to manipulate the camera or themselves to improve animage to be captured. For example, the systems described herein providereal-time (or near real-time) recommendations to adjust one or more oflighting conditions, camera angles, positioning of objects to becaptured, etc. Thus, one or more embodiments aid a user in capturing ahigh-quality image that mimics a target or template image. By guidingthe user to capture a digital image that matches one or morecharacteristics of a target or template image, the systems describedherein reduce the likelihood of capturing (and the overall number of)poor quality digital images and educate the user on how to capture highquality digital images in the future.

Additional features and advantages of the present application will beset forth in the description which follows, and in part will be obviousfrom the description, or may be learned by the practice of such exampleembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

This disclosure will describe one or more embodiments of the inventionwith additional specificity and detail by referencing the accompanyingfigures. The following paragraphs briefly describe those figures, inwhich:

FIG. 1 illustrates a schematic diagram of an example environment inwhich a smart photography system can operate in accordance with one ormore embodiments;

FIGS. 2A-2F illustrate an example user client device displaying exampleuser interfaces of a process of guiding a user to align image attributesof a camera feed with target image attributes in accordance with one ormore embodiments;

FIG. 3 illustrates an overview of an example process generating athree-dimensional face mesh in accordance with one or more embodiments;

FIG. 4 illustrates an example analysis of lighting of a digital image orcamera feed in accordance with one or more embodiments;

FIG. 5 illustrates an example flow of a step for providing a guide toinstruct a user to align image attributes of a camera feed with imageattributes of a target image model in accordance with one or moreembodiments;

FIG. 6 illustrates an example block diagram of the smart photographysystem in accordance with one or more embodiments;

FIG. 7 illustrates a flowchart of a series of acts for guiding a user toalign a camera feed with a target image model in accordance with one ormore embodiments; and

FIG. 8 illustrates a block diagram of an example computing device inaccordance with one or more embodiments.

DETAILED DESCRIPTION

One or more embodiments described herein provide benefits and solve oneor more of the foregoing or other problems in the art with a smartphotography system that guides a user to align one or more imageattributes of a camera feed with one or more image attributes of atarget image or target image model. In particular, the smart photographysystem described herein analyzes a camera feed of a user client deviceand compares image attributes of the camera feed with image attributesof a target image or target image model. Based on the comparison, thesmart photography system generates instructions for adjustments that auser can make to help align the camera feed with the target image ortarget image model.

To guide a user to capture a digital image that matches a target imageor target image model, the smart photography system displays, by way ofa user client device, a user interface that includes a live (e.g.,contemporaneous) camera feed. The smart photography system analyzes thecamera feed in real time to determine image attributes associated withthe camera feed. Based on those image attributes, the smart photographysystem compares the image attributes of the camera feed with predefined(e.g., predetermined) image attributes of the target image or targetimage model. Based on the comparison, the smart photography systemdetermines, in real time, changes that the user can make to adjust thecamera feed to have image attributes that are within a tolerance of theimage attributes associated with the target image or target image model.As the camera feed changes in response to adjustments made by the user,the smart photography system can update the analysis of the imageattributes and suggested adjustments. Accordingly, the smart photographysystem generates and provides a guide to instruct the user on steps toalign the image attributes of the camera feed with the image attributesof the target image or target image model.

As mentioned, the smart photography system provides a guide to help auser capture an image similar to a target image or target image model.For example, the smart photography system displays a number of targetimage models that the user may desire to mimic in a photograph. A targetimage model can comprise an analysis or indication of one or more targetimage attributes. In one or more embodiments, the smart photography canreceive a selection of a target image (e.g., another photograph) thatthe user desires to mimic. The smart photography system can thengenerate a target image model from the target image.

In response to a user selection of a target image model, the smartphotography system displays a target attribute overlay on the camerainterface that indicates areas including the target image attribute(s)from the target image model—which are therefore areas that the user alsowants to include the target image attribute(s) in the camera feed. Asthe user moves the camera (e.g., the user device), the smart photographysystem continuously updates the target attribute overlay to displaychanges in the target image attributes (e.g., lighting, shading,position of objects) within the camera feed in real time or near realtime.

Additionally, the smart photography system determines image attributesassociated with the camera feed. More specifically, based on generatinga three-dimensional mesh, the smart photography system determines imageattributes such as object position, object orientation (e.g., directionand tilt), and a direction of the light source(s) that illuminate thecamera feed. The smart photography system also displays an imageattribute overlay on the camera interface that indicates areas of thecamera feed including the image attribute(s). Thus, the smartphotography system can render attribute overlays together that bothillustrate target image attributes and current image attributes.

The smart photography system then compares the camera feed with aselected target image model. In particular, the smart photography systemcompares the image attributes of the camera feed with the known,predefined target image attributes of the selected target image model.Based on the comparison between the camera feed and the target imagemodel, the smart photography system generates a guide to align thecamera feed with the target image model. In particular, the smartphotography system determines how to correct differences between theimage attributes of the camera feed and the target image attributes ofthe target image model.

Upon determining how to correct the differences between the camera feedand the target image model, the smart photography system provides aguide to the user by way of the user client device. In particular, thesmart photography system displays arrows or other indicators to instructthe user to move the camera associated with the user device, objects inthe camera feed, and/or the light source. The smart photography systemmay further provide more detailed text-based instructions.

In response to detecting that the image attributes of the camera feedare within a tolerance (e.g., a similarity threshold) of the targetimage attributes of the target image model, the smart photography systemindicates this to the user. For example, the smart photography systemmay change the color of the attribute overlays or may display text(e.g., “Success!”) on the display screen to indicate to the user thatthe camera feed aligns with the target image model. In addition tonotifying the user that the camera feed matches the target image model,the smart photography system can further automatically capture one ormore digital images in response to detecting that the camera feed isaligned with the target image model. In other words, the smartphotography system can activate the camera to capture one or more imagesor capture screen shots of the user interface without user input tocapture an image (e.g., without the user selecting a capture button)based on detecting that the image attributes of the camera feed arewithin a tolerance of the target image attributes.

The smart photography system described herein provides a number ofadvantages over conventional digital image modification systems. Forexample, the smart photography system produces higher quality imagesthan conventional systems. In particular, because the smart photographysystem guides a user to align the camera feed with a target image model,the smart photography system prevents users from capturing poor qualitydigital images. Many conventional systems, on the other hand, arelimited in the quality of images they can produce because editingcapabilities can only overcome so many flaws in an originally-capturedlow-quality image.

In addition, the smart photography system is more computationallyefficient than conventional systems. Whereas conventional systemsrequire a significant amount of time, processing power, and othercomputer resources to edit digital images post-capture, the smartphotography system guides a user to align a camera feed with a targetimage model, thereby eliminating (or greatly reducing) the need for anypost-capture modifications. As a result, the smart photography systemuses fewer computer resources and less time than conventional systems toproduce high-quality images.

As another advantage, the smart photography system is also moreimmersive than conventional systems. In particular, the smartphotography system provides real-time (or near real-time) guidance withvisual overlays based on analysis of a camera feed. Many conventionalsystems, on the other hand, only provide the ability to edit digitalimages once they are captured, and therefore, do not provide animmersive experience. Thus, the smart photography system providesgreater flexibility over conventional post-capture modification systems.

Additional detail regarding the smart photography system is providedbelow. Throughout the description of the smart photography system, thisdisclosure will use various terminology to describe the systems,methods, and computer readable media associated with the smartphotography system. Accordingly, hereafter is provided a number ofdefinitions of terms that relate to the smart photography system and thedisclosure provided herein.

As used herein, the term “digital image” refers to a digitalrepresentation of objects captured by a camera device. For example, adigital image may refer to a digital photograph captured using a mobiledevice with an integrated digital camera. A digital image may also referto an image stored in a digital database accessible by a user clientdevice and/or a server. Furthermore, a digital image may portray a sceneincluding one or more faces (e.g., a selfie).

As used herein, a “target image model” refers to a model of a digitalimage with predefined or predetermined image attributes. Indeed, atarget image model generally refers to a model or representation of adigital image that a user may try to mimic or that has attributes that auser may try to reproduce in a different digital image. A target imagemodel can refer to a professional digital image with ideal lightingconditions (e.g., light source location), face position, faceorientation, and other image attributes. Indeed, a target image modelcan include known attributes such as a target lighting direction vector,a target face or head position, and a target face or head orientation(e.g., direction and/or tilt).

As mentioned, a user may capture a digital image by way of a user clientdevice. As used herein, the term “user client device” refers to acomputing device operable by a user. In particular, a user client devicecan refer to a computing device capable of capturing and/or storingdigital images. A user device can include a camera and can furtherinclude a display screen capable of displaying or presenting digitalcontent such as a digital image. For instance, a user client device caninclude a mobile device such as a smartphone, a tablet, or a wearabledevice.

As mentioned above, a user client device can present a camera feed byway of a display screen. As used herein, the term “camera feed” refersto a video representation of a scene captured by way of a cameraassociated with a user client device. A camera feed can refer to a videorepresentation simultaneously captured by way of the camera anddisplayed on a display screen of the user device. In addition, a camerafeed can refer to a scene captured and displayed in real time or nearreal time.

As further mentioned above, the smart photography system analyzes acamera feed to determine one or more image attributes associated withthe camera feed. As used herein, the term “image attributes” refers tofeatures or characteristics of a digital image. For example, imagepositioning attributes refer positions of objects within the digitalimage or camera feed (e.g., head or face position, head or faceorientation (e.g., direction and tilt), or positions of other objects ina camera feed). Image lighting attributes refer to a location or effectof one or more light sources, and/or shading that results from differentlight source locations. A light source location can comprise a vectorhaving a magnitude and a direction that indicates a location of a lightsource.

As will be described in further detail below, image attributes may referto camera feed attributes and/or to target image attributes (sometimesreferred to as target attributes). Camera feed attributes may refer tothe digital image attributes associated with the camera feed whiletarget image attributes may refer to the image attributes associatedwith the target image model.

In one or more embodiments, the smart photography system generates athree-dimensional mesh of a face depicted within the camera feed. Aswill be described in further detail below with reference to the figures,a “three-dimensional face mesh” refers to a three-dimensional mapping offacial landmarks and contours. In particular, a three-dimensional facemesh can map locations of facial landmarks such as a nose, mouth, eyes,hairline, etc.

As also mentioned, the smart photography system determines whether theimage attributes of the camera feed are within a tolerance of the imagelighting attributes of the target image model. As used herein, the term“tolerance” refers to a range of acceptable difference between one valueand another. A tolerance can refer to a threshold score or a thresholdvalue. For example, a tolerance can include a number of degrees of heador face tilt, a number of pixels of head movement in one direction oranother, a percentage of a shaded area in a camera feed as compared to atarget shaded area (e.g., the camera feed can have an 80% coverage ofthe shading in the target image model), or some other value.

More detail regarding the smart photography system will now be providedwith reference to the figures. For example, FIG. 1 illustrates aschematic diagram of an example environment 100 for implementing a smartphotography system 110 in accordance with one or more embodiments. Anoverview of the environment 100 is described in relation to FIG. 1.Thereafter, a more detailed description of the components and processesof the smart photography system 110 is provided in relation to thesubsequent figures.

As mentioned, FIG. 1 illustrates an exemplary environment 100 includinga network 102, server(s) 104, and a user client device 106 associatedwith a user 112. Each of the components of the environment 100 cancommunicate with via the network 102. Accordingly, the network 102 canfacilitate communications between the server(s) 104 and the user clientdevice 106 via appropriate network protocol. For example, the network102 may refer to a local network (e.g., a local area network or “LAN”),a wide area network (“WAN”), an Internet communication network, acellular network, a 3G or 4G network, or else may refer to differentcommunication protocol by which two computing devices can communicate.Example networks are discussed in more detail below with reference toFIG. 8.

As illustrated in FIG. 1, the environment 100 includes a server(s) 104.The server(s) 104 may refer to one or more computing devices that caninterface with the user client device 106 and can analyze digitalimages. In addition, while FIG. 1 illustrates the smart photographysystem 110 located on the user client device 106, in some embodimentsthe server(s) 104 include (e.g., implement) part of the entirety of thesmart photography system 110. The smart photography system 110 may beimplemented by and/or installed on the server(s) 104 as hardware,software, or both. Furthermore, as indicated by the dashed lines, insome embodiments the environment 100 does not include the server(s) 104.

As just mentioned, the environment 100 includes a user client device106. The user client device 106 is capable of communicating across thenetwork 102 to interface with the other components of the environment100 to receive and transmit data including digital images, camera feeds(e.g., via streaming), and image lighting attributes. Additionally, theuser client device 106 is capable of presenting, via a display, agraphical user interface (“GUI”) including a depiction of a camera feed.The user client device 106 is further capable of presenting a GUI thatincludes a camera feed in addition to a target image model, camera feedshading indicators, and/or target shading indicators.

As illustrated in FIG. 1, the user client device 106 includes an imagecapturing system 108. As used herein, the term “image capturing system”refers to hardware and/or software that enables a user client device(e.g., user client device 106) to capture, edit, store, share, orotherwise manage digital images. An image capturing system may refer toa webpage, a mobile application, a software program, executable hardwareinstructions, or a combination thereof. In some embodiments, the imagecapturing system 108 communicates with the server(s) 104 to uploaddigital images, download digital images, analyze digital images, and/orsynchronize digital images with a user account maintained by server(s)104. Alternatively, or additionally, the smart photography system 110can access target image model(s) or target images from the server(s)104.

As shown, the user client device 106 further includes the smartphotography system 110. In particular, the smart photography system 110may be part of the image capturing system 108 installed as hardware,software, or both. As will be described in further detail below, thesmart photography system 110 performs various functions, methods, andprocesses to guide a user to capture a digital image that matches (e.g.,with a tolerance) a target image model.

Although FIG. 1 illustrates a particular arrangement of the environment100 that includes the server(s) 104, the network 102, the smartphotography system 110, and the user client device 106, variousadditional or alternative arrangements are possible. For example, whileFIG. 1 illustrates the user client device 106 including the smartphotography system 110, in some embodiments the smart photography system110 may be located on the server(s) 104 and may communicate with theuser client device 106 via network 102.

As mentioned above, in one or more embodiments, the smart photographysystem 110 analyzes a camera feed captured by a user client device(e.g., user client device 106) to detect image attributes associatedwith the camera feed. In one or more embodiments, the smart photographysystem 110 analyzes image attributes of digital images and camera feedsrelative to one or more objects. For example, the smart photographysystem 110 can analyze image attributes relative to a face, head, orother object. The bulk of the description below of the smart photographysystem 110 is relative to the smart photography system 110 analyzingimage attributes relative to a face and providing guidance to help auser capture an image with a face having target image attributes. Onewill appreciate in light of the disclosure herein that the smartphotography system 110 can operate and provide guidance relative toother objects (e.g., inanimate objects, landscapes, animals, multiplefaces).

FIGS. 2A-2F illustrate representations of the user client device 106displaying a user interface 200 that includes a camera feed 202throughout the process of the smart photography system 110 analyzing thecamera feed 202 and guiding the user 112 to align the camera feed 202with a target image model. The description of FIGS. 2A-2F will providean overview of the smart photography system 110. Thereafter, a moredetailed description of the analysis of the camera feed 202 will beprovided in subsequent figures.

As illustrated in FIG. 2A, the smart photography system 110 analyzes thecamera feed 202 to identify a face depicted within the camera feed 202.In particular, the smart photography system 110 performs facial analysistechniques to identify facial contours and landmarks and fits a facemesh 204 to the face based on the analysis. Additional detail regardinggenerating the three-dimensional mesh of the face depicted within thecamera feed 202 is provided below with reference to FIG. 3. Furthermore,while FIG. 2A illustrates the camera feed 202 including a visualrepresentation of the face mesh 204, in some embodiments, the smartphotography system 110 displays the camera feed 202 without a visualrepresentation of the face mesh 204.

The smart photography system 110 further determines or estimates one ormore image attributes. For example, the smart photography system 110performs a light estimation technique to detect one or more lightsources that illuminate the camera feed 202 (or objects therein). In oneor more embodiments, the smart photography system 110 can generate athree-dimensional map of a face. The smart photography system 110 candetermine color values (e.g., RGB, HSV, etc.) of the face pixels withinthe camera feed 202. The smart photography system 110 further canfurther compare pixels to determine which are lighter and which aredarker. Furthermore, as part of analyzing the camera feed 202, the smartphotography system 110 can determine a location of the light source(s)based on comparing the pixels. For example, the smart photography system110 can generate a lighting vector to represent a location of a lightsource relative to a face within the camera feed 202. Along similarlines, the smart photography system 110 can generate (or access apreviously generated) a target lighting vector that represent a locationof a target light source from the target image model. Indeed, based onthe analysis of the camera feed 202, the smart photography system 110can generate a vector to designate the location of a light source (ormultiple light sources) that illuminates the camera feed 202 and candetermine how to manipulate the camera feed 202 and/or the face withinthe camera feed 202 to compensate for any differences between thelighting vector and the target lighting vector. Additional detailregarding determining the location of the light source(s) is providedbelow with reference to FIG. 4.

The smart photography system 110 further determines generates andprovides an image attribute overlay 208 in the user interface 202 thatrepresents the determined image attributes of the camera feed 202 in theuser interface 200 a. For example, utilizing the face mesh 204 and thelocation of the light source(s), the smart photography system 110generates a camera feed shading overlay 208 that indicates areas withinthe camera feed 202 that are shaded (e.g., areas on the face depictedwithin the camera feed 202 that are shaded). In particular, the smartphotography system 110 determines a face position and a face orientationof the face depicted within the camera feed 202 based on thethree-dimensional face mesh. Considering the face position andorientation, in conjunction with lighting vectors that indicate thelocation of light source(s), the smart photography system 110 determinesthe shaded areas within the camera feed 202. Indeed, as illustrated inFIG. 2A, the camera feed shading overlay 208 indicates that the lowerright portion of the face within the camera feed 202 is shaded.

While FIG. 2A illustrates the camera feed shading overlay 208 with aparticular lined shading pattern, the smart photography system 110 canprovide additional or alternative representations. For example, thesmart photography system 110 can provide a camera feed shading overlay208 can have a different pattern of lines, a different color, or someother appearance to indicate which portions of the camera feed 202 areshaded.

Furthermore, while FIG. 2A illustrates a single attribute overlay (e.g.,shading overlay 208), in alternative embodiments, the smart photographysystem 110 can generate multiple attribute overlays for different imageattributes (position, orientation, lighting, occlusion, etc.).Alternatively, the smart photography system 110 can provide a differentattribute overlay in place of the shading overlay 208.

As further illustrated in FIG. 2A, the user interface 200 a includes aselector gallery 206. In particular, the selector gallery 206 includes anumber of target image models which the user 112 can select. Based onthe selection of a target image model from the selector gallery 206, thesmart photography system 110 guides the user 112 to align the camerafeed 202 with the target image model. More specifically, the smartphotography system 110 generates a guide to instruct the user 112 tomanipulate the camera feed 202 (e.g., by moving the user device 106, thelocation of a light source, and/or the face depicted in the camera feed202) so that the image lighting attributes of the camera feed 202 arewithin a tolerance of the target image lighting attributes of theselected target image model. Additional detail regarding generating theguide and aligning the camera feed 202 with the target image model isprovided below in relation to subsequent figures.

Although FIG. 2A illustrates a particular number of selectable targetimage models within the selector gallery 206, in other embodiments theselector gallery 206 can include more or fewer target image models. Inaddition, the target image models can be three-dimensional renderings asdepicted in FIG. 2A, or the target image models can be digital images(e.g., photographs) that have ideal image attributes. For example, thetarget image models can be professional photographs that portray a facewith ideal shadows, face orientation, face position, etc. As shown, theselector gallery 206 includes nine target image models, each withdifferent image lighting attributes that are illustrated by thedifferent shading lines on each target image model.

The user can select a target image model from the selector gallery 206.In particular, the user 112 may select a target image model 210 (shownin FIG. 2B) from among the number of target image models in the selectorgallery 206 of FIG. 2A. For example, the user 112 may tap, touch, click,or otherwise provide user input to select the target image model 210 byway of a touchscreen or other input interface associated with the userclient device 106.

As mentioned above, the smart photography system 110 also generates andprovides a target attribute overlay. For example, as illustrated in FIG.2B, the smart photography system 110 provides a target shading overlay212. The target shading overlay 212 illustrates how or where the targetimage attribute would appear on, or relative, to the objects in thecamera feed 202. Indeed, as shown, the target shading overlay 212indicates shading on a portion of the face within the camera feed 202that corresponds to the shaded portion of the target image model 210.

While FIG. 2B illustrates a particular appearance of the target shadingoverlay 212, the smart photography system 110 can provide additional oralternative appearances for the target shading overlay 212. For example,the smart photography system 110 can provide the target shading overlay212 with a different lined pattern, a different color, or some otherappearance to indicate which portions of the camera feed 202 are to beshaded to match the target image model 210.

Additionally, the smart photography system 110 can configure a targetattribute overlay to have a different appearance than the imageattribute overlay. For example, the smart photography system 110 canconfigure the target and image attribute overlays to provide a visualdistinction of how an attribute currently affects, or is portrayed in, acamera feed (e.g., currently shaded portions of the camera feed 202)from where or how attribute would affect the camera feed if positionedor configured as in the target image model 210. For example, FIG. 2Billustrates an embodiment in which the smart photography system 110configures the shading overlay 208 with one appearance and the targetshading overlay 212 with a different appearance.

As mentioned above, to display the shading overlay 208, the smartphotography system 110 analyzes the camera feed 202 to determine acamera feed lighting vector that indicates a direction of a light sourcethat illuminates the camera feed 202. Accordingly, as the user 112 movesthe user client device 106 (the face within the camera feed 202 and/orthe light source that illuminates the camera feed 202), the smartphotography system 110 continuously updates the shading overlay 208 inreal time or near real time. Thus, the smart photography system 110 canchange the location of the shading overlay 208 within the camera feed202 as the image attributes change in response to movement of the userclient device 106, the face within the camera feed 202, and/or the lightsource.

Similarly, as the user 112 moves the user client device 106 (the facewithin the camera feed 202 and/or the light source that illuminates thecamera feed 202), the smart photography system 110 continuously updatesthe target shading overlay 212 in real time or near real time. Thus, thesmart photography system 110 can change the location of the targetshading overlay 212 within the camera feed 202 in response to movementof the user client device 106, the face within the camera feed 202,and/or the light source.

As mentioned previously, the smart photography system 110 can provideguidance on adjustments to make to align the image attributes with thetarget image attributes. For example, FIG. 2C illustrates that the smartphotography system 110 provides a user action button 213. In response todetecting a user interaction with (e.g., a selection of) the user actionbutton 213, the smart photography system 110 displays one or moreindicators of actions that the user 112 can perform to align the imageattributes of the camera feed 202 with target image attributes of thetarget image model 210. For example, in response to detecting a userinteraction with the user action button 213, the smart photographysystem 110 displays a lighting direction indicator 214 and a facedirection indicator 216. In particular, the lighting direction indicator214 indicates a direction in which the user 112 can move the lightsource to align the image lighting features of the camera feed 202 withthe target image lighting features of the target image model 210. Inother embodiments, the smart photography system 110 displays thelighting direction indicator 214 and the face direction indicator 216without first requiring a user interaction with the user action button213.

In addition, the lighting direction indicator 214 is interactive (e.g.,selectable). As described in further detail below with reference to FIG.2E, the user 112 can select the lighting direction indicator 214. Inresponse to receiving the user input to select the lighting directionindicator 214, the smart photography system 110 displays more detailedinstructions (e.g., text-based instructions) to guide the user 112through one or more actions to align the light source location of thecamera feed 202 with a light source location of the target image model210. By aligning the light source locations, the smart photographysystem 110 further aligns the camera feed shading overlay 208 and thetarget shading overlay 212.

Additionally, similar to the lighting direction indicator 214, the facedirection indicator 216 is also interactive. As described in furtherdetail below with reference to FIG. 2D, the smart photography system 110detects a user interaction with the face direction indicator 216. Inresponse to detecting the user interaction with the face directionindicator 216, the smart photography system 110 provides more detailedinstructions (e.g., text-based instructions) to guide the user 112through one or more actions to align the face position and/or faceorientation of the face depicted in the camera feed 202 with the faceposition and orientation of the target image model 210. By aligning theface position and orientation, in addition to the light sourcelocations, the smart photography system 110 further aligns the camerafeed shading overlay 208 and the target shading overlay 212.

The user interface 200 can further (or alternatively) include a cameradirection indicator. In particular, the camera direction indicator canindicate a direction in which the user 112 needs to move the cameraand/or the user client device 106 to align the camera feed 202 with thetarget image model 210. For example, the smart photography system 110can display a selectable camera direction indicator, and in response todetecting a user interaction with the camera direction indicator, thesmart photography system 110 can display more detailed instructions toguide the user through one or more actions to align the camera angleand/or camera position of the camera feed 202 with the target imagemodel 210.

As mentioned, the smart photography system 110 displays a guide toinstruct the user 112 to align the camera feed lighting attributes withthe target image lighting attributes. The guide can include the lightingdirection indicator 214, the face direction indicator 216, the cameradirection indicator, and/or detailed (e.g., text-based) instructions.For instance, FIG. 2D illustrates the user client device 106 displayingthe user interface 200 d that includes the camera feed 202, the facedirection indicator 216, and a detailed instruction 218.

As illustrated in FIG. 2D, the smart photography system 110 detects auser interaction with the face direction indicator 216, whereupon thesmart photography system 110 displays the detailed instruction 218,“Turn to the right.” Indeed, the smart photography system 110 providesmore detailed, text-based instructions to help the user 112 align thecamera feed 202 with the target image model 210 so that the camera feedlighting attributes match the target image lighting attributes (e.g.,within a tolerance). In particular, the detailed instruction 218corresponds to the face direction indicator 216. More specifically, thedetailed instruction 218 explains in words the instruction provided bythe face direction indicator 216. As shown, the detailed instruction 218instructs the user 112 to “Turn to the right.” Thus, the detailedinstruction 218 helps the user 112 more clearly understand how to alignthe camera feed 202 with the target image model 210.

As mentioned above, in some embodiments the smart photography system 110does not require user interaction with the face direction indicator 216before providing the detailed instruction 218. In particular, the smartphotography system 110 may instead provide the detailed instruction 218together with the face direction indicator 216 without first detecting auser interaction with the face direction indicator 216.

Additionally, the smart photography system 110 can include additional oralternative text instructions or guidance. For example, the smartphotography system 110 can provide instructions to turn in differentdirections, to move in different directions, to tilt in differentdirections, etc. Furthermore, the smart photography system 110 caninstruct the user 112 to move in varying degrees of intensity ordistance. To illustrate, the smart photography system 110 can provide adetailed instruction that instructs the user 112 “Turn slightly to theright,” or to “Turn sharply to the right,” or to “Tilt your chin upslightly,” or to “Tilt your head down a bit,” and so on.

In addition to providing varying degrees of detailed instructions, thesmart photography system 110 can further update the detailed instruction218 in real time. For example, as the user 112 moves the user clientdevice 106, the light source, and or the face within the camera feed202, the smart photography system 110 can periodically update thedetailed instruction 218 at regular intervals (e.g., as the smartphotography system 110 updates the camera feed shading overlay 208 ofFIG. 2C). Thus, as the user 112 turns to the right, the smartphotography system 110 may provide updated instructions along the linesof “Keep turning” or “Almost there.” Similarly, if the user 112 turnsthe wrong direction (e.g., to the left), the smart photography system110 can update the detailed instruction 218 to provide text such as“Turn the other way” or “This works, too! Keep turning.”

FIG. 2E illustrates the smart photography system 110 providing anadditional detailed instruction 220 in response to user adjustments.Similar to the discussion above in relation to the face directionindicator 216, the smart photography system 110 detects a userinteraction with the lighting direction indicator 214. In response tothe user interaction, the smart photography system 110 displays thedetailed instruction 220 in the form of a text overlay on thepresentation of the camera feed 202. As shown, the smart photographysystem 110 provides a detailed instruction 220 that says, “Move lightup.” Indeed, the smart photography system 110 instructs the user 112 tomove the location of the light source in an upward direction to alignthe camera feed shading overlay 208 with the target shading overlay 212.

As illustrated in FIG. 2E, the camera feed shading overlay 208 is on thesame side of the face as the target shading overlay 212. Morespecifically, as a result of the user 112 following the instructionsillustrated in FIG. 2D, the smart photography system 110 displays thecamera feed shading overlay 208 in FIG. 2E as more similar to the targetshading overlay 212 than in FIG. 2D—i.e., on the same side of the face.As a result of completing the instruction indicated by the facedirection indicator 216, the smart photography system 110 removes theface direction indicator 216 from the user interface 200 e. Inparticular, the smart photography system 110 determines that the facewithin the camera feed 202 is positioned and/or oriented within acertain tolerance (e.g., threshold) of the face within the target imagemodel 210. Based on that determination, the smart photography system 110removes the face direction indicator 216. In some embodiments, however,the smart photography system 110 does not remove the face directionindicator 216.

However, the camera feed shading overlay 208 of FIG. 2E is still notaligned with the target shading overlay 212. Indeed, as shown in FIG.2E, the camera feed shading overlay 208 covers only the bottom portionof the face within the camera feed 202, whereas the target shadingoverlay 212 covers most of the left side of the individual's face.Accordingly, the smart photography system 110 provides the lightingdirection indicator 214 to indicate to the user 112 to move the lightsource that illuminates the camera feed 202 upward. Additionally, asmentioned above, the smart photography system 110 provides the detailedinstruction 220 that clarifies or provides more explicit guidance to theuser 112. Similar to the discussion above in relation to the facedirection indicator 216, the smart photography system 110 may not firstdetect a user interaction with the lighting direction indicator 214before displaying the detailed instruction 220.

Although FIGS. 2D and 2E illustrate a particular order of events bywhich the smart photography system 110 guides the user 112 to align thecamera feed 202 with the target image model 210, additional oralternative operations are possible. For example, the processes involvedwith the description of FIG. 2E may take place before the processesdescribed in relation to FIG. 2D. To illustrate, the smart photographysystem 110 can detect a user interaction with the lighting directionindicator 214 and thereupon provide the corresponding instructionsbefore detecting a user interaction with the face direction indicator216.

Additionally, or alternatively, the smart photography system 110 canalternate between providing instructions pertaining to the lightingdirection indicator 214 and the face direction indicator 216 (and/or acamera direction indicator) without first requiring that the user 112complete one instruction before moving to the next. Indeed, the smartphotography system 110 can detect slight adjustments in lightingposition and face position throughout the process of guiding the user toalign the camera feed 202 with the target image model 210, and canupdate the detailed instructions and/or direction indicatorsaccordingly. Also, the smart photography system 110 can detect a userinteraction with the lighting direction indicator 214 before the user112 has finished the instruction associated with the face directionindicator 216. In these cases, the smart photography system 110 canpresent the detailed instruction 220 corresponding to the lightingdirection indicator 214 while still displaying the face directionindicator 216 to indicate to the user 112 that additional action isrequired with regards to the face position and/or orientation.

As mentioned, the smart photography system 110 guides the user 112 toalign the image attributes of the camera feed 202 with the target imageattributes of the target image model 210. For example, FIG. 2Fillustrates the user client device 106 displaying a user interface 200f. User interface 200 f includes the camera feed 202 and the targetimage model 210. In addition, user interface 200 f further includes thecamera feed shading overlay 208 and the target shading overlay 212. Asillustrated in FIG. 2F, the camera feed shading overlay 208 nearlymatches the target shading overlay 212. Indeed, FIG. 2F illustrates theuser interface 200 f depicting a situation where the user 112 hasadjusted the camera feed 202 by moving the user client device 106, theface within the camera feed 202, and/or the light source to align thelighting attributes of the camera feed 202 with the lighting attributesof the target image model 210.

As shown, the smart photography system 110 removes the lightingdirection indicator 214 and the face direction indicator 216 in responseto determining that the camera feed lighting attributes are within atolerance of the target image lighting attributes. Thus, in response todetecting that the face position and/or orientation within the camerafeed 202 is within a tolerance of the face position and/or orientationof the target image model 210, the smart photography system 110 removesthe face direction indicator 216. Similarly, in response to detectingthat the shading within the camera feed 202 is within a tolerance of theshading within the target image model 210, the smart photography system110 likewise removes the lighting direction indicator 214.

As mentioned above, the smart photography system 110 compares the camerafeed lighting attributes with the target image lighting attributes todetermine whether the camera feed lighting attributes are within atolerance of the target image lighting attributes. To compare thelighting attributes in this way, the smart photography system 110 maydetermine a location and a total area of the target image model 210 thatis shaded. The smart photography system 110 may further determine alocation and a total area of the face within the camera feed 202 that isshaded. The smart photography system 110 can further determine whetherthe shaded area of the camera feed 202 is within a threshold area (e.g.,a number of square pixels) of the shaded area of the target image model210. The smart photography system 110 may further determine whether theshaded area within the camera feed 202 is within a threshold distance ofthe shaded portion of the target image model 210 based on, for examplethe midpoints of each respective shaded areas.

Upon determining that the image attributes of the camera feed 202 arewithin a tolerance of the target image attributes of the target imagemodel 210, the smart photography system 110 provides a match indicator222. In particular, the match indicator 222 indicates to the user 112that the camera feed 202 matches the target image model 210, at leastwithin an acceptable range of error. As shown, the smart photographysystem 110 provides a text-based match indicator, “Success!” In someembodiments, however, the smart photography system 110 provides atext-based match indicator with different text. In the same or otherembodiments, the smart photography system 110 provides a match indicatorby changing the color of the camera feed shading overlay 208 and/or thetarget shading overlay 212 (e.g., turning them both green).Additionally, or alternatively, the smart photography system 110 changesthe color of the target image model 210 (e.g., by highlighting thetarget image model 210, changing the color of the border of the targetimage model 210, or by some other way) to indicate to the user 112 thatthe camera feed 202 matches the target image model 210. The smartphotography system 110 can additionally or alternatively provide anaudio indicator and/or a haptic (e.g., vibration) indicator as well.

As mentioned above, the smart photography system 110, when dealing withselfies, can generate a three-dimensional mesh of a face depicted withinthe camera feed 202. In particular, the smart photography system 110analyzes the camera feed 202 to identify and track various faciallandmarks and contours. For instance, the smart photography system 110utilizes machine learning model techniques to analyze the camera feed202 and generate the three-dimensional mesh. As used herein, the term“machine learning model” refers to a computational model that can betuned (e.g., trained) based on inputs to approximate unknown functions.In particular, the term machine-learning model can include a model thatuses machine learning algorithms to learn to approximate complexfunctions and generate outputs based on a plurality of inputs (e.g., atraining dataset including a plurality of digital images classified intoscene categories). As used herein, a machine-learning model can include,but is not limited to, a neural network (e.g., a convolutional neuralnetwork or deep learning), decision tree, association rule learning,inductive logic programming, support vector learning, Bayesian network,regression-based model, principal component analysis, or a combinationthereof.

While FIGS. 2A-2F illustrate various functions of the smart photographysystem 110 in relation to selfies, in some embodiments the smartphotography system 110 can similarly operate using the front-facingcamera to capture objects other than portraits (e.g., full body images,landscapes). To illustrate, the smart photography system 110 can analyzea camera feed that includes more of an individual than just a face tocompare image attributes relative to, for example, a full body shotshown within the camera feed. Indeed, the smart photography system 110can compare the camera feed with a target image of a body in accordancewith the disclosure herein. For example, the smart photography system110 can compare attributes of the camera feed with target attributes ofthe target image to provide a guide on how to align lighting,positioning (e.g., positioning of hands or limbs), or other aspects ofthe camera feed with the target image.

In addition to the foregoing, the smart photography system 110 canoperate in a two-person mode where a first user operates a computingdevice and aims a camera of the computing device at a second user. Inparticular, the computing device may include more than one camera (e.g.,a front camera and a back camera), and the smart photography system 110may be able to communicate with either or both cameras of the computingdevice to perform the methods described herein. Indeed, the smartphotography system 110 may interface with a front camera when dealingwith selfies and may interface with a back camera when dealing withtraditional photographs in two-person mode.

Furthermore, in some embodiments the smart photography system 110 canperform the processes and functions described herein in relation to morethan a single face within a camera feed. For example, based on thetechniques described herein, the smart photography system 110 cananalyze multiple faces in a group portrait to provide a guide on how toalign each face within the camera feed with a target image model.Alternatively, the smart photography system 110 can provide a guide onhow to align the group of faces as a whole with a target image modelthat also includes an image of a group of images.

In addition, while FIGS. 2A-2F relate to providing a guide on how toalign attributes of the camera feed 202 with a target image model 210,in some embodiments the smart photography system 110 can perform thesefunctions automatically. In particular, the smart photography system 110can interface with a drone or other autonomous system that performsoperations or manipulations to adjust the camera feed 202. Accordingly,as the smart photography system 110 analyzes the camera feed 202 todetermine how to align the camera feed 202 with the target image 210,the smart photography system 110 can instruct a drone holding (orincluding) the camera capturing the camera feed 202 to move to achievealignment with the target image model 210. For example, the smartphotography system 110 can instruct the drone to move horizontally,vertically, to tilt, rotate, or to perform a combination thereof toalign the camera feed 202 with the target image model 210.

As illustrated in FIG. 3, the smart photography system 110 generates athree-dimensional face mesh 306 based on a morphable model 304. Themorphable model 304 is a generic, three-dimensional model of a face thatincludes a number of vertices (e.g., illustrated by the intersections ofgridlines of the morphable model 304) that define various points on theface. Thus, while FIG. 3 illustrates the morphable model 304 including aparticular number of vertices, in some embodiments the morphable modelcan include more or fewer vertices. Additional detail regardinggenerating the morphable model 304 is provided below.

Additionally, the smart photography system 110 analyzes the face 302 toidentify salient points 308 and further aligns the morphable model 304to the salient points 308. Thus, the smart photography system 110generates the face mesh 306 to look like the face depicted within thecamera feed 202 described above based on the identified salient points308. Although FIG. 3 illustrates a simplified face mesh 306 thatincludes fewer vertices than the morphable model 304, in someembodiments the face mesh 306 can include more or fewer vertices (e.g.,the same number of vertices as the morphable model 304).

To generate the face mesh 306 based on the morphable model 304 as wellas the salient points 308, the smart photography system 110 implements amulti-linear principal component analysis (“PCA”) model. In particular,the first two dimensions of the PCA model represent a facial identity(e.g., geometric shape and skin reflectance) and the third dimensionrepresents the facial expression. Accordingly, the smart photographysystem 110 can parameterize a given face within the camera feed 202 as:

_(geo)(α,δ)=α_(id) +E _(id) *α+E _(exp)*δ

and

_(alb)(β)=α_(alb) +E _(alb)*β.

This parameterization assumes a multivariate normal probabilitydistribution of shape and reflectance around the average shape α_(id)∈

^(3n) and reflectance α_(alb)∈

^(3n). The shape E_(id)∈

^(3n×80), reflectance E_(alb)∈

^(3n×80), and expression E_(exp)∈

^(3n×76) basis and the corresponding standard deviations σ_(id) ∈

⁸⁰, σ_(alb) ∈

⁸⁰, and σ_(exp)∈

⁷⁶ are given. The PCA model has 53,000 vertices and 106,000 faces. Insome embodiments, however, the PCA model can include more or fewervertices and/or faces. The smart photography system 110 generates asynthesized image

by rasterizing the PCA model under a rigid model transformation Φ(ν) andthe full perspective transformation II(ν).

Given a monocular input sequence, the smart photography system 110 canreconstruct all unknown parameters

jointly with a robust variational optimization. The proposed object ishighly non-linear with regard to the unknown parameters and has thefollowing components:

E(

)=ω_(col) E _(col)(

)+ω_(lan) E _(lan)(

)+ω_(reg) E _(reg)(

)

where ω_(col)E_(col)(

)+ω_(lan)E_(lan)(

) is the data term and ω_(reg) E_(reg) (

) is the prior term.

The data term measures the similarity between the synthesized imageryand the input data in terms of photo-consistency E_(col) and facialfeature alignment E_(lan). The smart photography system 110 takes thelikelihood of a given parameter vector

into account by utilizing the statistical regularizer E_(reg). Theweights ω_(col), ω_(lan), and ω_(reg) balance the three differentsub-objectives. In some embodiments, the smart photography system 110predefines the weights and sets each respective weight at, for example,ω_(col)=1, ω_(lan)=10, and ω_(reg)=2.5*10⁻⁵.

To quantify how well the input data is explained by a synthesized image,the smart photography system 110 measures the photo-metric alignmenterror on pixel level:

${E_{col}()} = {\frac{1}{}{\sum\limits_{p \in }{{{_{}(p)} - {_{}(p)}}}_{2}}}$

where

is the synthesized image,

is the input RGB image, and p∈V denote all visible pixel positions in

.

The smart photography system 110 uses an l_(2,1)-norm to be robustagainst outliers. In particular, distance in color space is based on l₂,while the smart photography system 110 uses an l₁-norm in the summationover the all pixels. In other embodiments, however, the smartphotography system 110 may use a least-squares formulation.

In addition, the smart photography system 110 enforces featuresimilarity between a set of salient points pairs (e.g., taken from thesalient points 308) detected in the RGB stream:

${E_{lan}()} = {\frac{1}{\mathcal{F}}{\sum\limits_{f_{j} \in \mathcal{F}}{\omega_{{conf},j}{{f_{j} - {{II}\left( {\Phi \left( _{j} \right)} \right)}}}_{2}^{2}}}}$

To this end, the smart photography system 110 employs a facial landmarktracking algorithm. In particular, the smart photography system 110 canimplement the facial landmark tracking algorithm set forth in Jason M.Saragih, Simon Lucey, and Jeffrey F. Cohn, Deformable model fitting byregularized landmark mean-shift, International Journal of ComputerVision 91.2 200-215 (2011), which is incorporated herein by reference inits entirety. Based on this facial landmark tracking technique, eachfeature point f_(j) ∈

⊂

² comes with a detection confidence ω_(conf,j) and corresponds to aunique vertex ν_(j)=

_(geo)(α, δ)∈

³ of the face prior. Thus, the smart photography system 110 avoidsissues with local minima in the highly-complex energy landscape ofE_(col)(

).

The smart photography system 110 further enforces plausibility of thesynthesized faces based on the assumption of a normal distributedpopulation. In particular, the smart photography system 110 enforces theparameters to stay statistically close to the mean:

${E_{reg}()} = {{\sum\limits_{i = 1}^{80}\; \left\lbrack {\left( \frac{\alpha_{i}}{\sigma_{{id},i}} \right)^{2} + \left( \frac{\beta_{i}}{\sigma_{{alb},i}} \right)^{2}} \right\rbrack} + {\sum\limits_{i = 1}^{76}{\left( \frac{\delta_{i}}{\sigma_{\exp,i}} \right)^{2}.}}}$

By using this regularization strategy, the smart photography system 110prevents degeneration of the facial geometry and reflectance, and alsoguides the optimization strategy out of the local minima.

As described, and as illustrated in FIG. 3, the smart photography system110 generates the face mesh 306 based on the morphable model 304 andfurther based on analyzing the face 302 to identify salient points 308.Although the above description with reference to FIG. 3 includes variousequations and formulas, in some embodiments the smart photography system110 can implement one or more of the techniques and processes describedin Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt,and Matthias NieBner, Face2face: Real-time face capture and reenactmentof rgb videos, Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, 2387-2395 (2016), which is incorporated herein byreference in its entirety.

As mentioned, the smart photography system 110 can further determine anumber and location of light sources that illuminate the camera feed202. To illustrate the process of determining light source locations,FIG. 4 shows an object 400, a light source 402 that causes the shading412, and a target light source 404 that causes the shading 414. Forillustrative purposes, the discussion of FIG. 4 relates to the object400, the light source 402, and the target light source 404. However, thedescription of FIG. 4 relates to the above discussion in relation to thecamera feed 202 and the target image model 210.

In particular, the light source 402 is analogous to a detected lightsource that illuminates the camera feed 202 described above. Thus, theshading 412 is analogous to the camera feed shading overlay 208described above. Likewise, the target light source 404 is analogous tothe light source associated with the target image model 210, and thetarget shading 414 is analogous to the target shading overlay 212, alsodescribed above. Accordingly, FIG. 4 illustrates an example environmentin which the smart photography system 110 can analyze the illuminationof the object 400 to determine lighting vectors and thereby determineany adjustments necessary to align the light source 402 with the targetlight source 404.

Indeed, as shown in FIG. 4, the smart photography system 110 detects alight source 402 that illuminates the object 400, thereby causing theshading 412. More specifically, the smart photography system 110 candetermine a light source vector 406 that indicates a direction of thelight source 402. Additionally, the smart photography system 110accesses known (e.g., predetermined) parameters including a location ofa target light source 404 that causes a target shading 414. Indeed, thesmart photography system 110 can analyze a target image with a targetlight source 404 to generate the known parameters. Thus, the smartphotography system 110 accesses a target light source vector 408 thatindicates a direction and/or distance of the target light source 404.

Accordingly, based on the light source vector 406 and the target lightsource vector 408, the smart photography system 110 determines adifference between the location of the light source 402 and the targetlight source 404. To determine this difference, the smart photographysystem 110 can calculate a correction vector 410 that indicates adirection and distance required to move the light source 402 to matchthe location of the target light source 404.

In particular, the smart photography system 110 implements, for thelight source 402 as well as the target light source 404, a lightingdetection model suited for natural scene illumination. The smartphotography system 110 combines both depth and intensity cues of thecamera feed 202 (e.g., as captured as RGB-D data by the camera of theuser client device 106). The lighting model accounts for light sources,multiple albedos, and local lighting effects such as specularities,shadows, and inter-reflections.

To illustrate, the smart photography system 110 implements a shadingfunction that relates a surface geometry to its intensity image. Thesmart photography system 110 analyzes the image under naturalillumination where there is no single point light source. In particular,the smart photography system 110 employs an extended intrinsic imagedecomposition model for recovery intrinsic images. The smart photographysystem 110 can efficiently incorporate this model to generate a surfacereconstruction:

L(i,j,{right arrow over (n)})=ρ(i,j)S({right arrow over (n)})+β(i,j)

where L(i,j,{right arrow over (n)}) is the image lighting at each pixel,S({right arrow over (n)}) is the shading, ρ(i,j) accounts for multiplescene albedos and shadowed areas by adjusting the shading intensity. Inaddition, β(i, j) is an independent, spatially changing, near lightsource that accounts for local lighting variations such asinter-reflections and specularities. As used hereafter, the (i,j)indexing nomenclature is sometimes omitted for convenience and ease ofinterpretation.

Initially, the smart photography system 110 assumes a Lambertian sceneto recovery the shading S associated with light sources that have auniform effect on a given image (e.g., the camera feed 202 and/or thetarget image model 210). Once the smart photography system 110 computesthe shading, the smart photography system 110 finds ρ and β to betterexplain the intensity image based on the object geometry. In someembodiments, the smart photography system 110 sets ρ=1 and β=0 duringthe shading recovery process.

Generally, the irradiance of diffuse objects in natural illuminationscenes can be well described by low order spherical harmonicscomponents. Thus, the smart photography system 110 implements a smoothfunction to recover the shading image. For efficiency and simplicity,the smart photography system 110 uses zero and first order sphericalharmonics which are a linear polynomial of a surface normals and areindependent of a given pixel's location. Therefore, they can berepresented by:

S({right arrow over (n)})={right arrow over (m)} ^(T) ñ

where {right arrow over (n)} is the surface normal, S(n) is the shadingfunction, {right arrow over (m)} is a vector of the four first orderspherical harmonics coefficients, and ñ=({right arrow over (n)},1)^(T).

Every valid pixel in the aligned intensity image I can be used torecover the shading. Hence, the smart photography system 110 generatesan over-determined least squares parameter estimation problem:

argmin_({right arrow over (m)}) =∥{right arrow over (m)} ^(T) ñ−I∥ ₂ ².

The rough normals that the smart photography system 110 obtains from theinitial depth map eliminate the need for assumptions and constraints onthe shape as well as the need for using several images. Thus, despitehaving only the normals of the smoothed surface, the smart photographysystem 110 can still obtain an accurate shading model. In addition, theestimated surface normals eliminate the need for pre-calibrating thesystem lighting, thereby enabling the smart photography system 110 tofunction in dynamic lighting environments.

The shading alone provides only a rough assessment overall lighting. Toaccount for specularities, shadows, and nearby light sources, the smartphotography system 110 further calculates ρ and β. Indeed, based on thedetermination of S (described above), the smart photography system 110can then recover ρ.

To recover ρ, the smart photography system 110 freezes S to the shadingimage and optimizes ρ to distinguish between the scene albedos, and toaccount for shadows (β is still set to 0, as its recovery is describedbelow). The smart photography system 110 sets a fidelity term tominimize the l₂ error between the proposed model (e.g., the target imagemodel 210) and the input image (e.g., the camera feed 202). However, toavoid over-fitting, the smart photography system 110 utilizes a priorterm that prevents ρ from changing too rapidly. Thus, the smartphotography system 110 implements a model that explains only lightingchanges and not geometry changes.

Following the retinex theory and other similar intrinsic image recoveryalgorithms, the smart photography system 110 assumes that the albedo mapis piecewise smooth and that there is a low number of albedos in theimage. Furthermore, the smart photography system 110 utilizes a weightedLaplacian to distinguish between materials and albedos on a scene, whilealso maintaining a smooth-changing nature of light. The penalty term fordoing so may be represented as:

${{\sum\limits_{k \in }{\omega_{k}^{c}{\omega_{k}^{d}\left( {\rho - \rho_{k}} \right)}}}}_{2}^{2}$

where

is the neighborhood of the pixel, ω_(k) ^(c) is an intensity weightingterm:

$\omega_{k}^{c} = \left\{ \begin{matrix}{0,} & {{{I_{k} - I}}_{2}^{2} > \tau} \\{{\exp \left( {- \frac{{{I_{k} - {I\left( {i,j} \right)}}}_{2}^{2}}{2\sigma_{c}^{2}}} \right)},} & {otherwise}\end{matrix} \right.$

and ω_(k) ^(d) is the following depth weighting term:

$\omega_{k}^{d} = {\exp \left( {- \frac{{{z_{k} - {z\left( {i,j} \right)}}}_{2}^{2}}{2\sigma_{d}^{2}}} \right)}$

where σ_(d) is a parameter responsible for the allowed depthdiscontinuity and z(i,j) represents the depth value of the respectivepixel.

Using this regularization term, the smart photography system 110performs a three-dimensional segmentation of a scene (e.g., a scenedepicted by the camera feed 202), dividing into piecewise smooth parts.Therefore, the smart photography system 110 accounts for material andalbedo changes, but smart photography system 110 smooths subtle changesin the surface. In sum, the smart photography system 110 generates tofollowing regularized linear least squares problem with respect to ρ:

${\min_{\rho}{{{\rho \; {S\left( \overset{\rightarrow}{n} \right)}} - I}}_{2}^{2}} + {\lambda_{\rho}{{{\sum\limits_{k \in }{\omega_{k}^{c}{\omega_{k}^{d}\left( {\rho - \rho_{k}} \right)}}}}_{2}^{2}.}}$

As mentioned above, after the smart photography system 110 calculates ρ,the smart photography system 110 then calculates β. The smartphotography system 110 can implement a similar recovery process for β asthe smart photography system 110 uses for ρ. However, since first orderspherical harmonics account for approximately 87.5% of scene lighting,the smart photography system 110 also limits the energy of β to beconsistent with the shading model. Thus, the smart photography system110 determines β by:

$\min_{\rho}{{\beta - \left( {I - {\rho \; {S\left( \overset{\rightarrow}{n} \right)}} - I} \right._{2}^{2} + {\lambda_{\rho}{{{\sum\limits_{k \in }{\omega_{k}^{c}{\omega_{k}^{d}\left( {\rho - \rho_{k}} \right)}}}}_{2}^{2}.}}}}$

Accordingly, the smart photography system 110 generates a lightingestimation model for the camera feed 202 and/or the target image model210. Based on generating both a lighting model for the camera feed 202and the target image model 210, the smart photography system 110 candetermine a difference between lighting locations of the camera feed 202and the target image model 210. Thus, the smart photography system 110can generate a guide to correct the differences and align the lightingattributes of the camera feed 202 with the lighting attributes of thetarget image model 210.

Although the above description with reference to FIG. 4 includes variousequations and formulas for estimating light source locations, in someembodiments the smart photography system 110 can implement one or moreof the techniques and processes described in Roy Or-El, Guy Rosman,Aaron Wetzler, Ron Kimmel, and Alfred M. Bruckstein, RGBD-fusion:Real-time high precision depth recovery, Proceedings of the IEEEConference on Computer Vision and Pattern, Recognition, 5407-5416(2015), which is incorporated herein by reference in its entirety.

As mentioned above, the smart photography system 110 provides a guide toinstruct a user (e.g., user 112) to align a camera feed (e.g., camerafeed 202) with a target image model (e.g., target image model 210).Indeed, FIG. 5 illustrates a step for providing a guide to instruct auser associated with a user client device to align image attributes of acamera feed with target image attributes of a target image model. Thestep for providing a guide to instruct a user associated with a userclient device to align image attributes of a camera feed with targetimage attributes of a target image model can include the belowdescription of FIG. 5, in addition to relevant methods and techniquesdescribed elsewhere in this disclosure, including but not limited to,the equations and formulas described in relation to FIGS. 3 and 4.

FIG. 5 illustrates a flow 500 that includes a number of acts 502-512included in the above-mentioned step for providing a guide to instruct auser associated with a user client device to align image attributes of acamera feed with target image attributes of a target image model. Inparticular the flow 500 includes an act 502 whereby the smartphotography system 110 analyzes a camera feed. For example, the smartphotography system 110 analyzes the camera feed by implementing thealgorithms and processes described above to determine camera feedlighting attributes and target image lighting attributes. Additionaldetail regarding the camera feed lighting attributes and the targetimage lighting attributes is provided above.

Based on the analysis of the camera feed, the smart photography system110 further performs act 504 to compare camera feed lighting attributeswith the target image lighting attributes. Indeed, the smart photographysystem 110 compares the image lighting attributes by determining adifference in lighting position between the camera feed and the targetimage model. The smart photography system 110 further determines adifference in face position and orientation between the camera feed andthe target image model. Additional detail regarding these determinationsis provided above in relation to previous figures.

As shown, the smart photography system 110 further performs act 506 todetermine adjustments to align the camera feed with the target imagemodel. In particular, the smart photography system 110 calculates acorrection vector necessary to align the light source location(s) of thecamera feed with the light source location(s) of the target image model.In addition, the smart photography system 110 determines a number ofpixels and/or degrees that a user needs to move a face shown within thecamera feed to match the position and orientation of the face shownwithin the target image model.

As further illustrated in FIG. 5, the smart photography system 110performs act 508 to provide instructions to the user to adjust thecamera feed. In particular, the smart photography system 110 displays alighting direction indicator to guide the user to adjust the lightsource location to align the light source location of the camera feedwith that of the target image model. Further, the smart photographysystem 110 provides a face direction indicator to guide the user to movethe position and/or orientation of the face shown within the camerafeed. As mentioned above, the smart photography system 110 may further(or alternatively) provide a camera direction indicator to guide theuser to move the camera of the user client device to align the imagelighting features of the camera feed with those of the target imagemodel. Additional detail regarding the lighting direction indicator, theface direction indicator, and the camera direction indicator is providedabove.

FIG. 5 further illustrates act 510 whereby the smart photography system110 determines whether the camera feed lighting attributes are within atolerance of the target image lighting attributes. As described above,the smart photography system 110 determines whether the face shownwithin the camera feed is oriented within a threshold number of degreesin a vertical direction and a horizontal direction of the face withinthe target image model. The smart photography system 110 furtherdetermines whether the light location of the camera feed (and thereforethe shading of the camera feed) is within a threshold number of degreesof the light location of the target image model. The smart photographysystem 110 can further determine whether the shading on the face incamera feed is within a threshold pixel area or threshold percentage ofthe shading on the face within the target image model.

In response to determining that the camera feed lighting attributes arewithin a tolerance of the target image lighting attributes, the smartphotography system 110 performs act 512 to provide a match indicator. Inparticular, the smart photography system 110 provides an indication tothe user that the camera feed is sufficiently aligned with the targetimage model. As described above, the indication can include a text-basedindication, a change in overlay colors, an audio indicator, and/or ahaptic indicator.

In response to determining that the camera feed lighting attributes arenot within a tolerance of the target image lighting attributes, on theother hand, the smart photography system 110 repeats acts 502 through510. Indeed, the smart photography system 110 can continuously repeatacts 502-510 numerous times over as the camera feed refreshes (e.g.,once per frame, once per three frames, once per five frames, etc.) toconstantly analyze the camera feed to detect when the camera feed alignswith the target image model to satisfy act 510.

Looking now to FIG. 6, additional detail will be provided regardingcomponents and capabilities of a smart photography system 602. The smartphotography system 602 of FIG. 6 may be the same as the smartphotography system 110 of FIG. 1. Specifically, FIG. 6 illustrates anexample schematic diagram of the smart photography system 110 on anexample computing device 600 (e.g., the server(s) 104 and/or the userclient device 106, a drone, a camera, a smart phone). As shown in FIG.6, the smart photography system 110 may include a GUI manager 604, auser input detector 606, a face analyzer 608, a light estimator 610, animage lighting attribute comparison manager 612, and a storage manager614. While FIG. 6 depicts a particular number of components, in someembodiments, the smart photography system 110 may include more or fewercomponents. In addition, the components may perform additional oralternative tasks than those described hereafter.

As mentioned, the smart photography system 602 includes a GUI manager604. The GUI manager 604 may present, display, provide, or otherwisemanage a GUI associated with the smart photography system 602. Forexample, the GUI manager 604 can present a GUI that includes a camerafeed, a target image model, a lighting direction indicator, a facedirection indicator, a camera direction indicator, a user action button213, and/or other elements. To illustrate, the GUI manager 604 cancommunicate with an image capturing device (e.g., a camera) associatedwith the computing device 600 to display a captured camera feed. The GUImanager 604 can also present detailed instructions (e.g., text) as wellas any visible indicators described above (e.g., the match indicator).

As also mentioned, the smart photography system 602 includes a userinput detector 606. In particular, the user input detector 606 canreceive or detect any input or user interaction with the computingdevice 600. For example, the user input detector 606 can detect userinput by way of a touchscreen or some other input interface by which auser can provide user input to interact with a GUI of the smartphotography system 602. In response to detecting user input in relationto a particular element of the GUI, the user input detector 606 canfurther communicate with the face analyzer 608, the image lightingattribute comparison manager 612, the GUI manager 604, and/or some othercomponent of the smart photography system 602 to cause performance ofone or more of the processes described above.

As shown, the smart photography system 602 includes a face analyzer 608.In particular, the face analyzer 608 can analyze a face within a camerafeed and/or a target image model to identify or detect facial contoursand landmarks. For example, the face analyzer 608 can implement theprocesses and algorithms described above, and can communicate with theimage lighting attribute comparison manager 612 to determine differences(e.g., differences in position and/or orientation) between a face withinthe camera feed and a face within the target image model. Based on anydetermined differences, the face analyzer 608 can communicate with theGUI manager 604 to present GUI elements to guide a user to align thecamera feed with the target image model.

As illustrated in FIG. 6, the smart photography system 602 furtherincludes a light estimator 610. In particular, the light estimator 610can analyze a camera feed and/or a target image model to determinelighting attributes associated with the camera feed and/or the targetimage model. For example, the light estimator 610 can implement thelighting attribute algorithms described above. In addition, the lightestimator 610 can communicate with the image lighting attributecomparison manager 612 to compare the lighting attributes associatedwith the camera feed with the lighting attributes associated with thetarget image model.

Indeed, the image lighting attribute comparison manager 612 can comparethe lighting attributes associated with the camera feed with thelighting attributes associated with a target image model. Indeed, theimage lighting attribute comparison manager 612 can compare faceposition, face orientation, light source position, etc. Based on thecomparison, the image lighting attribute comparison manager 612 cancommunicate with the GUI manager 604 to provide instructions to guide auser to align the lighting attributes of the camera feed with thelighting attributes of the target image model.

As also illustrated in FIG. 6, the smart photography system 602 includesa storage manager 614. In particular, the storage manager 614 caninclude a target image database 616 and can store, maintain, orotherwise manage target image models. For example, the target imagedatabase 616 can include a number of target image models to includewithin a selector gallery, as described above. The target image database616 can further store image lighting attributes associated with eachstored target image model. Thus, the storage manager 614 can communicatewith the image lighting attribute comparison manager 612 to compare thestored image lighting attributes of a selected target image model withthe determined image lighting attributes of a camera feed.

FIGS. 1-6, the corresponding text, and the examples provide a number ofdifferent systems and methods that analyze a camera feed to determineimage lighting attributes and compare the determined image lightingattributes with corresponding image lighting attributes associated withthe target image model to generate and provide instructions to guide auser to align the camera feed with the target image model. In additionto the foregoing, embodiments can also be described in terms offlowcharts comprising acts for accomplishing a particular result. Forexample, FIG. 7 illustrates a flowchart of an exemplary series of actsin accordance with one or more embodiments.

While FIG. 7 illustrates acts according to one embodiment, alternativeembodiments may omit, add to, reorder, and/or modify any of the actsshown in FIG. 7. The acts of FIG. 7 can be performed as part of amethod. Alternatively, a non-transitory computer readable medium cancomprise instructions, that when executed by one or more processors,cause a computing device to perform the acts of FIG. 7. In still furtherembodiments, a system can perform the acts of FIG. 7. Additionally, thesteps/acts described herein may be repeated or performed in parallelwith one another or in parallel with different instances of the same orother similar steps/acts.

FIG. 7 illustrates an exemplary series of acts 700 for guiding a user toalign a camera feed with a target image model. In particular, the seriesof acts 700 includes an act 702 of capturing a live camera feed. Forexample, the act 702 can involve capturing a live camera feed via acomputing device.

As shown, the series of acts 700 includes an act 704 of determining animage attribute. In particular, the act 704 can involve determining animage attribute of the camera feed. Although not illustrated in FIG. 7,the series of acts 700 can further include an act of generating, basedon the camera feed, a three-dimensional mesh of a face depicted withinthe camera feed, wherein the three-dimensional mesh includes a mappingof the face depicted within the camera feed. Thus, the act 704 canfurther involve determining, based on the camera feed, a location of alight source that illuminates the camera feed and determining, based onthe three-dimensional mesh, a face position of the face depicted withinthe camera feed.

Determining the location of the light source that illuminates the camerafeed can include calculating a normalized lighting vector for the lightsource that illuminates the camera feed. In addition, comparing theimage attribute of the camera feed with the target image attribute ofthe target image model can include determining a difference between thenormalized lighting vector and a predefined normalized target lightingvector associated with the target image model.

As further shown in FIG. 7, the series of acts 700 includes an act 706of comparing the image attribute with a target image attribute. Inparticular, the act can involve comparing the image attribute of thecamera feed with a target image attribute of a target image model. Thetarget image attribute of the target image model can include one or moreof a target head position, a target light source location, or a targetcamera position.

In addition, the series of acts 700 includes an act 708 of providing auser interface with instructions to align the image attribute with thetarget image attribute. In particular, the act 708 can involve, based onthe comparison, providing, by way of a user interface on the computingdevice, a guide with interactive, real-time instructions on how to alignthe image attribute of the camera feed with the target image attributeof the target image model. The act 708 can further involve displaying atarget shading indicator that indicates shading within the target imagemodel and a camera feed shading indicator that indicates shading withinthe camera feed. Displaying the target shading indicator can includedisplaying a target shade overlay on the camera feed to indicate atarget shading associated with the target image model. Additionally, oralternatively, displaying the target shading indicator can includeproviding a rendering of the target image model that comprises a targetshade texture that indicates target shading associated with the targetimage model.

In addition, the act 708 can involve providing, to the user by way ofthe user client device, instructions on how to manipulate the camerafeed to align the camera feed shading indicator with the target shadingindicator. Displaying the camera feed shading indicator can includedisplaying a camera feed shade overlay on the camera feed to indicate acurrent shading of the camera feed. Additionally, or alternatively,displaying the camera feed shading indicator can include overlaying acamera feed shade texture on the camera feed to indicate a currentshading of the camera feed. The instructions on how to manipulate thecamera feed can include one or more of a face direction indicator thatindicates a direction to move a face depicted within the camera feed ora lighting direction indicator that indicates a direction to move alight source that illuminates the camera feed.

Though not illustrated in FIG. 7, the series of acts 700 can include anact of determining that the image attribute associated with the camerafeed is within a tolerance of the target image attribute of the targetimage model. The series of acts can also include an act of providing, byway of the user client device and in response to determining that theimage attribute of the camera feed is within the tolerance of the targetimage attribute of the target image model, an alignment indicator thatindicates an alignment of the camera feed with respect to the targetimage model.

Furthermore, the series of acts 700 can include an act of providing, byway of the user interface on the user client device, a plurality oftarget image models as well as an act of receiving a user input toselect a target image model from among the plurality of target imagemodels. The series of acts 700 can further include an act of displaying,by way of the user client device, a detailed instruction element, and anact of, in response to detecting a user selection of the detailedinstruction element, providing detailed instructions to the user on howto align the camera feed with the target image model.

The series of acts 700 can also include an act of, in response todetecting that the camera feed shading indicator is within an alignmenttolerance of the target shading indicator, providing a match indicatorto alert the user that the camera feed is aligned with the target imagemodel.

Embodiments of the present disclosure may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. Embodiments within the scope of the presentdisclosure also include physical and other computer-readable media forcarrying or storing computer-executable instructions and/or datastructures. In particular, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices (e.g., any of the media content access devicesdescribed herein). In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory, etc.), and executes those instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein.

Computer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arenon-transitory computer-readable storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,embodiments of the disclosure can comprise at least two distinctlydifferent kinds of computer-readable media: non-transitorycomputer-readable storage media (devices) and transmission media.

Non-transitory computer-readable storage media (devices) includes RAM,ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM),Flash memory, phase-change memory (“PCM”), other types of memory, otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium which can be used to store desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media tonon-transitory computer-readable storage media (devices) (or viceversa). For example, computer-executable instructions or data structuresreceived over a network or data link can be buffered in RAM within anetwork interface module (e.g., a “NIC”), and then eventuallytransferred to computer system RAM and/or to less volatile computerstorage media (devices) at a computer system. Thus, it should beunderstood that non-transitory computer-readable storage media (devices)can be included in computer system components that also (or evenprimarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general-purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. In someembodiments, computer-executable instructions are executed on ageneral-purpose computer to turn the general-purpose computer into aspecial purpose computer implementing elements of the disclosure. Thecomputer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, and the like. The disclosuremay also be practiced in distributed system environments where local andremote computer systems, which are linked (either by hardwired datalinks, wireless data links, or by a combination of hardwired andwireless data links) through a network, both perform tasks. In adistributed system environment, program modules may be located in bothlocal and remote memory storage devices.

Embodiments of the present disclosure can also be implemented in cloudcomputing environments. In this description, “cloud computing” isdefined as a model for enabling on-demand network access to a sharedpool of configurable computing resources. For example, cloud computingcan be employed in the marketplace to offer ubiquitous and convenienton-demand access to the shared pool of configurable computing resources.The shared pool of configurable computing resources can be rapidlyprovisioned via virtualization and released with low management effortor service provider interaction, and then scaled accordingly.

A cloud-computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. Acloud-computing model can also expose various service models, such as,for example, Software as a Service (“SaaS”), Platform as a Service(“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud-computingmodel can also be deployed using different deployment models such asprivate cloud, community cloud, public cloud, hybrid cloud, and soforth. In this description and in the claims, a “cloud-computingenvironment” is an environment in which cloud computing is employed.

FIG. 8 illustrates, in block diagram form, an exemplary computing device800 that may be configured to perform one or more of the processesdescribed above. One will appreciate that the smart photography system110 can comprise implementations of the computing device 800. As shownby FIG. 8, the computing device can comprise a processor 802, memory804, a storage device 806, an I/O interface 808, and a communicationinterface 810. Furthermore, the computing device 800 can include acamera and a display screen. In certain embodiments, the computingdevice 800 can include fewer or more components than those shown in FIG.8. Components of computing device 800 shown in FIG. 8 will now bedescribed in additional detail.

In particular embodiments, processor(s) 802 includes hardware forexecuting instructions, such as those making up a computer program. Asan example, and not by way of limitation, to execute instructions,processor(s) 802 may retrieve (or fetch) the instructions from aninternal register, an internal cache, memory 804, or a storage device806 and decode and execute them.

The computing device 800 includes memory 804, which is coupled to theprocessor(s) 802. The memory 804 may be used for storing data, metadata,and programs for execution by the processor(s). The memory 804 mayinclude one or more of volatile and non-volatile memories, such asRandom-Access Memory (“RAM”), Read Only Memory (“ROM”), a solid-statedisk (“SSD”), Flash, Phase Change Memory (“PCM”), or other types of datastorage. The memory 804 may be internal or distributed memory.

The computing device 800 includes a storage device 806 includes storagefor storing data or instructions. As an example, and not by way oflimitation, storage device 806 can comprise a non-transitory storagemedium described above. The storage device 806 may include a hard diskdrive (HDD), flash memory, a Universal Serial Bus (USB) drive or acombination of these or other storage devices.

The computing device 800 also includes one or more input or output(“I/O”) devices/interfaces 808, which are provided to allow a user toprovide input to (such as user strokes), receive output from, andotherwise transfer data to and from the computing device 800. These I/Odevices/interfaces 808 may include a mouse, keypad or a keyboard, atouch screen, camera, optical scanner, network interface, modem, otherknown I/O devices or a combination of such I/O devices/interfaces 808.The touch screen may be activated with a writing device or a finger.

The I/O devices/interfaces 808 may include one or more devices forpresenting output to a user, including, but not limited to, a graphicsengine, a display (e.g., a display screen), one or more output drivers(e.g., display drivers), one or more audio speakers, and one or moreaudio drivers. In certain embodiments, devices/interfaces 808 isconfigured to provide graphical data to a display for presentation to auser. The graphical data may be representative of one or more graphicaluser interfaces and/or any other graphical content as may serve aparticular implementation.

The computing device 800 can further include a communication interface810. The communication interface 810 can include hardware, software, orboth. The communication interface 810 can provide one or more interfacesfor communication (such as, for example, packet-based communication)between the computing device and one or more other computing devices 800or one or more networks. As an example, and not by way of limitation,communication interface 810 may include a network interface controller(MC) or network adapter for communicating with an Ethernet or otherwire-based network or a wireless NIC (WNIC) or wireless adapter forcommunicating with a wireless network, such as a WI-FI. The computingdevice 800 can further include a bus 812. The bus 812 can comprisehardware, software, or both that couples components of computing device800 to each other.

In the foregoing specification, the invention has been described withreference to specific exemplary embodiments thereof. Various embodimentsand aspects of the invention(s) are described with reference to detailsdiscussed herein, and the accompanying drawings illustrate the variousembodiments. The description above and drawings are illustrative of theinvention and are not to be construed as limiting the invention.Numerous specific details are described to provide a thoroughunderstanding of various embodiments of the present invention.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. For example, the methods described herein may beperformed with less or more steps/acts or the steps/acts may beperformed in differing orders. Additionally, the steps/acts describedherein may be repeated or performed in parallel with one another or inparallel with different instances of the same or similar steps/acts. Thescope of the invention is, therefore, indicated by the appended claimsrather than by the foregoing description. All changes that come withinthe meaning and range of equivalency of the claims are to be embracedwithin their scope.

What is claimed is:
 1. A non-transitory computer readable medium forguiding a user to capture a digital image that matches a target imagecomprising instructions that, when executed by a processor, cause acomputer device to: capture a live camera feed via a computing device;determine an image attribute of the camera feed; compare the imageattribute of the camera feed with a target image attribute of a targetimage model; and based on the comparison, provide, by way of a userinterface on the computing device, a guide with interactive, real-timeinstructions on how to align the image attribute of the camera feed withthe target image attribute of the target image model.
 2. Thenon-transitory computer readable medium of claim 1, wherein the targetimage attribute of the target image model comprises one or more of atarget head position, a target light source location, or a target cameraposition.
 3. The non-transitory computer readable medium of claim 1,further comprising instructions that, when executed by the processor,cause the computer system to generate, based on the camera feed, athree-dimensional mesh of a face depicted within the camera feed,wherein the three-dimensional mesh comprises a mapping of the facedepicted within the camera feed.
 4. The non-transitory computer readablemedium of claim 3, wherein the instructions cause the computer device todetermine the image attribute of the camera feed by: determining, basedon the camera feed, a location of a light source that illuminates thecamera feed; and determining, based on the three-dimensional mesh, aface position of the face depicted within the camera feed.
 5. Thenon-transitory computer readable medium of claim 4, wherein determiningthe location of the light source that illuminates the camera feedcomprises calculating a normalized lighting vector for the light sourcethat illuminates the camera feed.
 6. The non-transitory computerreadable medium claim 5, wherein the instructions cause the computer todevice to compare the image attribute of the camera feed with the targetimage attribute of the target image model by determining a differencebetween the normalized lighting vector and a predefined normalizedtarget lighting vector associated with the target image model.
 7. Thenon-transitory computer readable medium of claim 1, further comprisinginstructions that, when executed by the processor, cause the computerdevice to: determine that the image attribute of the camera feed iswithin a tolerance of the target image attribute of the target imagemodel; and provide, by way of the computing device and in response todetermining that the image attribute of the camera feed is within thetolerance of the target image attribute of the target image model, analignment indicator that indicates an alignment of the camera feed withrespect to the target image model.
 8. The non-transitory computerreadable medium of claim 1, further comprising instructions that, whenexecuted by the processor, cause the computer device to: provide, by wayof the user interface on the computing device, a plurality of targetimage models; and receive a user input to select a target image modelfrom among the plurality of target image models.
 9. In a digital mediumenvironment for capturing digital images, a system for guiding a user tocapture a digital image that matches a target image, the systemcomprising: a camera; at least one processor; and a non-transitorycomputer readable medium comprising instructions that, when executed bythe at least one processor, cause the system to: capture a live camerafeed via the camera; determine an image attribute of the camera feed byanalyzing the camera feed to identify a light source of the camera feed;compare the image attribute of the camera feed with a target imageattribute of a target image model by determining a difference betweenthe light source of the camera feed and a target light source; and basedon the comparison, provide, by way of a user interface, a guide withinteractive, real-time instructions on how to align the image attributeof the camera feed with the target image attribute of the target imagemodel.
 10. The system of claim 9, wherein the target image attribute ofthe target image model comprises one or more of a target head position,a target light source location, or a target camera position.
 11. Thesystem of claim 9, further comprising instructions that, when executedby the at least one processor, cause the system to: provide, by way ofthe user interface on the computing device, a plurality of target imagemodels; and receive a user input to select one of the plurality oftarget image models.
 12. The system of claim 9, wherein the instructionscause the system to provide the guide on how to align the imageattribute of the camera feed with the target image attribute by:displaying a target shading indicator that indicates shading within thetarget image model and a camera feed shading indicator that indicatesshading within the camera feed; and providing, to the user by way of thecomputing device, instructions on how to manipulate the camera feed toalign the camera feed shading indicator with the target shadingindicator.
 13. The system of claim 12, wherein: displaying the targetshading indicator comprises displaying a target shade overlay within thecamera feed to indicate a target shading relative to a current view ofthe camera feed; and displaying the camera feed shading indicatorcomprises displaying a camera feed shade overlay within the camera feedto compare with the target shade overlay.
 14. The system of claim 12,wherein: displaying the target shading indicator comprises generating arendering of the target image model to display within the user interfaceand that comprises a target shade texture that indicates target shadingof the target image model; and displaying the camera feed shadingindicator comprises overlaying a camera feed shade texture on the camerafeed to indicate a current shading of the camera feed to compare withthe rendering of the target image model.
 15. The system of claim 12,wherein the instructions on how to manipulate the camera feed compriseone or more of: a face direction indicator that indicates a direction tomove a face depicted within the camera feed; or a lighting directionindicator that indicates a direction to move a light source thatilluminates the camera feed.
 16. The system of claim 12, furthercomprising instructions that, when executed by the at least oneprocessor, cause the system to, in response to detecting that the camerafeed shading indicator is within an alignment tolerance of the targetshading indicator, provide a match indicator to alert the user that thecamera feed is aligned with the target image model.
 17. The system ofclaim 9, further comprising instructions that, when executed by the atleast one processor, cause the system to: display, by way of thecomputing device, a detailed instruction element; and in response todetecting a user selection of the detailed instruction element, providedetailed instructions to the user on how to align the camera feed withthe target image model.
 18. In a digital medium environment forcapturing digital images, a computer-implemented method for guiding auser to capture a digital image that matches a target image, the methodcomprising: analyzing a selection of a target image model to identify atarget image attribute; analyzing a live camera feed to identify animage attribute; and performing a step for providing a guide on how toalign the image attribute of the camera feed with the target imageattribute of the target image model.
 19. The computer-implemented methodof claim 18, further comprising: providing, by way of a user interfaceon a computing device, a plurality of target image models; and receivinga user input to select a target image model from among the plurality oftarget image models.
 20. The computer-implemented method of claim 18,further comprising, in response to detecting that the camera feed isaligned with the target image model, automatically capturing a digitalimage without user input.